Systems and methods for contextual searching of semantic entities

ABSTRACT

A method for contextual searching of a semantic entity includes the steps of: receiving a rich query comprising a semantic entity to be searched and contextual information regarding the selection of the semantic entity; receiving, from a search engine, a plurality of search results returned in response to submitting a search request using the selected semantic entity; selecting a plurality of content sources to be searched based on the returned search results; converting the received rich query into a plurality of simple queries, each one of the plurality of simple queries corresponding to one of the plurality of content sources; submitting each one of the plurality of simple queries to the corresponding content source; receiving results from each of the plurality of content sources; and filtering the plurality of received results into a single result list for presentation to a user.

RELATED APPLICATION

This present application claims the benefit of and priority to U.S.Provisional Patent Application No. 61/512,826, entitled “SYSTEMS ANDMETHODS FOR CONTEXTUAL PERSONALIZED SEARCHING,” filed on Jul. 28, 2011,which is incorporated by reference in its entirety.

FIELD OF THE INVENTION

Embodiments of the present invention relate generally to the field ofsearching the internet and other content sources (including, but notlimited to, advertising streams) and computerized searching. Inparticular, the systems and methods described herein allow traditionalweb content to be consumed in a radically different manner based oncharacteristics of the consumer.

SUMMARY OF THE INVENTION

In one aspect, the present invention relates to a method for contextualsearching of a semantic entity. A rich query comprising a semanticentity to be searched and contextual information regarding the selectionof the semantic entity is received. A plurality of search resultsgenerated by a search engine is returned in response to submitting asearch request using the selected semantic entity. A plurality ofcontent sources to be searched is selected based on the returned searchresults. The received rich query is converted into a plurality of simplequeries, each one of the plurality of simple queries corresponding toone of the plurality of content sources. Each one of the plurality ofsimple queries is submitted to the corresponding content source andresults are received from each of the plurality of content sources. Theplurality of received results is filtered into a single result list forpresentation to a user.

In another aspect, the present invention relates to a system forcontextual searching of a semantic entity that includes: means forreceiving a rich query comprising a semantic entity to be searched andcontextual information regarding the selection of the semantic entity;means for receiving, from a search engine, a plurality of search resultsreturned in response to submitting a search request using the selectedsemantic entity; means for selecting a plurality of content sources tobe searched based on the returned search results; means for convertingthe received rich query into a plurality of simple queries, each one ofthe plurality of simple queries corresponding to one of the plurality ofcontent sources; means for submitting each one of the plurality ofsimple queries to the corresponding content source; means for receivingresults from each of the plurality of content sources; and means forfiltering the plurality of received results into a single result listfor presentation to a user.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, aspects, features, and advantages ofthe disclosure will become more apparent and better understood byreferring to the following description taken in conjunction with theaccompanying drawings, in which:

FIG. 1A is a schematic diagram of a system in accordance with anembodiment of the present invention;

FIG. 1B is a exploded diagram of the apparatus of FIG. 1, in accordancewith an embodiment of the present invention;

FIG. 2 is a schematic diagram of a system transferring information, inaccordance with an embodiment of the present invention;

FIG. 3A is an exemplary view of a screen shot showing a webpage beforecontextual mode has been activated, according to an embodiment of thepresent invention;

FIGS. 3B and 3C are exemplary views of screen shots showing a webpageafter contextual mode has been activated and various search terms arehighlighted, according to some embodiments of the present invention;

FIG. 3D is an exemplary view of a screen shot showing an overlay windowand a webpage after a contextual search has been conducted, according toan embodiment of the present invention;

FIG. 4 is a flow diagram of a process according to an embodiment of thepresent invention;

FIGS. 5A-5D are more detailed flow diagrams of a process according to anembodiment of the present invention.

FIG. 6 is a textual representation of a data structure containinginformation regarding a sample user profile, according to an embodimentof the present invention;

FIG. 7A is an exemplary view of a screen shot showing a webpage in whicha user has highlighted a search entity;

FIG. 7B is a sample rich-query string based on the search entity andwebpage of FIG. 7A, according to an embodiment of the present invention;and

FIG. 7C is a textual representation of a data structure containing asample text search result as returned by a server that had been providedthe rich-query string of FIG. 7B, according to an embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

Various embodiments of the present invention provide a process or systemfor conducting a contextual search. A user views a webpage, a document,or other content. According to some embodiments, the contextual searchservice accepts input from the user regarding one or more search terms,phrases, images, or other information indicating a topic or topics ofinterest (hereinafter referred to as a “search entity”). According tosome embodiments, anything and everything in a document is searchable.According to various embodiments, the service selects additional inputrelated to the search entity to determine the context of the searchentity, in order to conduct a search that is more likely to be relevantthan a search that does not involve the context. According to variousfurther embodiments, the service uses the search entity and context thatmay be related to the search entity to search for and obtain searchresults. According to various embodiments, these results are presentedto the user.

FIG. 1A illustrates a networked environment 101 according to someembodiments, in which a contextual search service is provided. As shownin FIG. 1A, the networked environment 101 includes one or more clientmachines 102A-102N (generally referred to herein as “client machine(s)102” or “client(s) 102”) in communication with one or more servers106A-106N (generally referred to herein as “server machine(s) 106” or“server(s) 106”) over a network 104. The client machine(s) 102 can, insome embodiments, be referred to as a single client machine 102 or asingle group of client machines 102, while server(s) 106 may be referredto as a single server 106 or a single group of servers 106. Althoughthree client machines 102 and three server machines 106 are depicted inFIG. 1A, any number of clients 102 can be in communication with anynumber of servers 106. In some embodiments, a single client machine 102communicates with more than one server 106, while in another embodimenta single server 106 communicates with more than one client machine 102.In yet another embodiment, a single client machine 102 communicates witha single server 106. Further, although a single network 104 is shownconnecting client machines 102 to server machines 106, it should beunderstood that according to some embodiments, multiple, separatenetworks connect a subset of client machines 102 to a subset of servermachines 106.

In some embodiments, the computing environment 101 includes an appliance(not shown in FIG. 1A) installed between the server(s) 106 and clientmachine(s) 102. This appliance can mange client/server connections, andin some cases can load balance connections made by client machines 102to server machines 106. Suitable appliances are manufactured by any oneof the following companies (but are not limited to these companies): theCitrix Systems Inc. Application Networking Group; Silver Peak Systems,Inc, both of Santa Clara, Calif.; Riverbed Technology, Inc. of SanFrancisco, Calif.; F5 Networks, Inc. of Seattle, Wash.; or JuniperNetworks, Inc. of Sunnyvale, Calif.

Client(s) 102 and server(s) 106 may be provided as a computing device100, a specific embodiment of which is illustrated in FIG. 1B. Includedwithin the computing device 100 is a system bus 150 that communicateswith the following components: a central processing unit 121(“processor”); a main memory 122 providing non-transient storage;storage memory 128 providing non-transient storage; an input/output(I/O) controller 123; display devices 124A-124N; an installation device116; and a network interface 118. In some embodiments, the storagememory 128 includes: an operating system, software routines, and aclient agent 120. The I/O controller 123, in some embodiments, isfurther connected one or more input devices. Volatile computer memory,non-volatile computer memory, and combinations of volatile andnon-volatile computer memory are to be included within the scope ofnon-transient storage. As shown in FIG. 1B, the I/O controller 123 isconnected to a camera 125, a keyboard 126, a pointing device 127 such as(but not limited to) a mouse, and a microphone 129.

Various embodiments of the computing machine 100 include a centralprocessing unit 121 characterized by any one of the following componentconfigurations: logic circuits that respond to and process instructionsfetched from the main memory unit 122; a microprocessor unit, such as(but not limited to): those manufactured by Intel Corporation; thosemanufactured by Motorola Corporation; those manufactured by TransmetaCorporation of Santa Clara, Calif.; the RS/6000 processor such as thosemanufactured by International Business Machines; a processor such asthose manufactured by Advanced Micro Devices; or any other combinationof logic circuits. Still other embodiments of the central processingunit 122 include any one or combination of the following: amicroprocessor, a microcontroller, a central processing unit with asingle processing core, a central processing unit with two processingcores, or a central processing unit with more than one processing core.

While FIG. 1B illustrates a computing device 100 that includes a singlecentral processing unit 121, in some embodiments the computing device100 include one or more processing units 121. In these embodiments, thecomputing device 100 can store and execute firmware or other executableinstructions that, when executed, direct the one or more processingunits 121 to simultaneously execute instructions or to simultaneouslyexecute instructions on a single piece of data. In other embodiments,the computing device 100 store and execute firmware or other executableinstructions that, when executed, direct the one or more processingunits to each execute a section of a group of instructions. For example,each processing unit 121 may be instructed to execute a portion of aprogram or a particular module within a program.

In some embodiments, the processing unit 121 includes one or moreprocessing cores. For example, according to some embodiments theprocessing unit 121 has two cores, four cores, eight cores, or anynumber of cores. In some embodiments, the processing unit 121 comprisesone or more parallel processing cores. The processing cores of theprocessing unit 121, in some embodiments, access available memory as aglobal address space, or in other embodiments, memory within thecomputing device 100 is segmented and assigned to a particular corewithin the processing unit 121. In some embodiments, the one or moreprocessing cores or processors in the computing device 100 can eachaccess local memory. In still other embodiments, memory within thecomputing device 100 is shared amongst one or more processors orprocessing cores, while other memory can be accessed by particularprocessors or subsets of processors. In embodiments where the computingdevice 100 includes more than one processing unit, the multipleprocessing units are included in a single integrated circuit (IC). Thesemultiple processors, in some embodiments, are linked together by aninternal high speed bus, which may be referred to as an elementinterconnect bus.

In embodiments where the computing device 100 includes one or moreprocessing units 121, or a processing unit 121 including one or moreprocessing cores, the processors, in some embodiments, can execute asingle instruction simultaneously on multiple pieces of data (SIMD), orin other embodiments can execute multiple instructions simultaneously onmultiple pieces of data (MIMD). In some embodiments, the computingdevice 100 includes any number of SIMD and MIMD processors.

The computing device 100, in some embodiments, includes a graphicsprocessor or a graphics processing unit (not shown). In variousembodiments, the graphics processing unit includes any combination ofsoftware and hardware, and can further input graphics data and graphicsinstructions, render a graphic from the inputted data and instructions,and output the rendered graphic. In some embodiments, the graphicsprocessing unit is included within the processing unit 121. In otherembodiments, the computing device 100 includes one or more processingunits 121, such that at least one processing unit 121 is dedicated toprocessing and rendering graphics.

Some embodiments of the computing device 100 provide support for any oneof the following installation devices 116 (but is not limited to thesedevices): a CD-ROM drive, a CD-R/RW drive, a DVD-ROM drive, tape drivesof various formats, USB device, a bootable medium, a bootable CD, abootable CD for GNU/Linux distribution such as KNOPPIX®, a hard-drive orany other device suitable for installing applications or software.Applications, in some embodiments, include a client agent 120, or anyportion of a client agent 120. According to some embodiments, thecomputing device 100 further includes a storage device 128 that iseither one or more hard disk drives, or one or more redundant arrays ofindependent disks; where the storage device is configured to store anoperating system, software, programs applications, or at least a portionof the client agent 120. A further embodiment of the computing device100 includes an installation device 116 that is used as the storagedevice 128.

Embodiments of the computing device 100 include any one of the followingI/O devices 130A-130N (but is not limited to these examples): a camera125, keyboard 126; a pointing device 127; a microphone 129; mice;trackpads; an optical pen; trackballs; microphones; drawing tablets;video displays; speakers; inkjet printers; laser printers; anddye-sublimation printers; touch screen; or any other input/output deviceable to perform the methods and systems described herein. According tosome embodiments, an I/O controller 123 connects to multiple I/O devices103A-130N to control the one or more I/O devices. According to someembodiments, the I/O devices 130A-130N are configured to provide storageor an installation medium 116. According to other embodiments, the I/Odevices 130A-130N are configured to provide a universal serial bus (USB)interface for receiving USB storage devices such as (but not limited to)the USB Flash Drive line of devices manufactured by Twintech Industry,Inc. Still other embodiments include an I/O device 130 that is a bridgebetween the system bus 150 and an external communication bus, such as(but not limited to): a USB bus; an Apple Desktop Bus; an RS-232 serialconnection; a SCSI bus; a FireWire bus; a FireWire 800 bus; an Ethernetbus; an AppleTalk bus; a Gigabit Ethernet bus; an Asynchronous TransferMode bus; a HIPPI bus; a Super HIPPI bus; a SerialPlus bus; a SCI/LAMPbus; a FibreChannel bus; or a Serial Attached small computer systeminterface bus.

In some embodiments, the computing machine 100 can execute any operatingsystem, while in other embodiments the computing machine 100 can executeany of the following operating systems: versions of the MICROSOFTWINDOWS operating systems such as WINDOWS 3.x; WINDOWS 95; WINDOWS 98;WINDOWS 2000; WINDOWS NT 3.51; WINDOWS NT 4.0; WINDOWS CE; WINDOWS XP;WINDOWS VISTA; and WINDOWS 7; the different releases of the Unix andLinux operating systems; any version of the MAC OS or iOS manufacturedby Apple Computer; OS/2, manufactured by International BusinessMachines; any embedded operating system; any real-time operating system;any open source operating system; any proprietary operating system; anyoperating systems for mobile computing devices; or any other operatingsystem. In still another embodiment, the computing machine 100 canexecute multiple operating systems. For example, the computing machine100 can execute PARALLELS or another virtualization platform that canexecute or manage a virtual machine executing a first operating system,while the computing machine 100 executes a second operating systemdifferent from the first operating system.

In various embodiments, the computing machine 100 is embodied in any oneof the following computing devices (but is not limited to thesedevices): a computing workstation; a desktop computer; a laptop ornotebook computer; a server; a handheld computer; a mobile telephone; aportable telecommunication device; a media playing device; a gamingsystem; a mobile computing device; a netbook; a device of the IPODfamily of devices manufactured by Apple Computer; any one of thePLAYSTATION family of devices manufactured by the Sony Corporation; anyone of the Nintendo family of devices manufactured by Nintendo Co; anyone of the XBOX family of devices manufactured by the MicrosoftCorporation; or any other type and/or form of computing,telecommunications or media device that is capable of communication andthat has sufficient processor power and memory capacity to perform themethods and systems described herein.

In other embodiments, the computing machine 100 is a mobile device suchas (but not limited to) any one of the following mobile devices: aJAVA-enabled cellular telephone or personal digital assistant (PDA),such as the i55sr, i58sr, i85s, i88s, i90c, i95cl, or the iml100, all ofwhich are manufactured by Motorola Corp; the 6035 or the 7135,manufactured by Kyocera; the i300 or i330, manufactured by SamsungElectronics Co., Ltd; the TREO 180, 270, 600, 650, 680, 700p, 700w, or750 smart phone manufactured by Palm, Inc; any computing device that hasdifferent processors, operating systems, and input devices consistentwith the device; or any other mobile computing device capable ofperforming the methods and systems described herein. In still otherembodiments, the computing device 100 is any one of the following mobilecomputing devices (but is not limited to these devices): any one seriesof Blackberry, or other handheld device manufactured by Research InMotion Limited; the iPhone manufactured by Apple Computer; Palm Pre; aPocket PC; a Pocket PC Phone; or any other handheld mobile device. Inyet still other embodiments, the computing device 100 is a smart phoneor tablet computer, such as (but not limited to) the iPhone or iPadmanufactured by Apple, Inc. of Cupertino, Calif.; the BlackBerry devicesmanufactured by Research in Motion, Ltd. of Waterloo, Ontario, Canada;Windows Mobile devices manufactured by Microsoft Corp., of Redmond,Wash.; the Xoom manufactured by Motorola, Inc. of Libertyville, Ill.;devices capable of running the Android platform provided by Google, Inc.of Mountain View, Calif.; or any other type and form of portablecomputing device.

In still other embodiments, the computing device 100 is a virtualmachine. In some embodiments, the virtual machine is any virtualmachine, such as (but not limited to) those managed by a hypervisordeveloped by XenSolutions, Citrix Systems, IBM, VMware, or any otherhypervisor. In still other embodiments, the virtual machine is managedby a hypervisor executing on a server 106 or a hypervisor executing on aclient 102.

In still other embodiments, the computing device 100 can execute,operate or otherwise provide an application that is any one of thefollowing: software; an application or program; executable instructions;a virtual machine; a hypervisor; a web browser; a web-based client; aclient-server application; an ActiveX control; a Java applet; softwarerelated to voice over internet protocol (VoIP) communications like asoft IP telephone; an application for streaming video and/or audio orreceiving and playing streamed video and/or audio; an application forfacilitating real-time-data communications; a HTTP client; a FTP client;or any other set of executable instructions. Still other embodimentsinclude a client device 102 that displays application output generatedby an application remotely executing on a server 106 or other remotelylocated machine. In these embodiments, the client device 102 can displaythe application output in an application window, a browser, or otheroutput window.

According to various embodiments, the computing device 100 furtherincludes a network interface 118 to interface to a Local Area Network(LAN), Wide Area Network (WAN), the Internet, or any type of network,through a variety of connections including, but not limited to, standardtelephone lines, LAN or WAN links (e.g., 802.11, T1, T3, 56 kb, X.25,SNA, DECNET), broadband connections (e.g., ISDN, Frame Relay, ATM,Gigabit Ethernet, Ethernet-over-SONET), wireless connections, or somecombination of any or all of the above. Connections can also beestablished using a variety of communication protocols (e.g., TCP/IP,IPX, SPX, NetBIOS, Ethernet, ARCNET, SONET, SDH, Fiber Distributed DataInterface (FDDI), RS232, RS485, IEEE 802.11, IEEE 802.11a, IEEE 802.11b,IEEE 802.11g, CDMA, GSM, WiMax and direct asynchronous connections). Insome embodiments, the network 104 comprises one or more sub-networks,and is installed between any combination of the clients 102, servers106, computing machines and appliances included within the computingenvironment 101. In some embodiments, the network 104 is: a local-areanetwork (LAN); a metropolitan area network (MAN); a wide area network(WAN); a primary network 104 comprised of multiple sub-networks 104located between the client machines 102 and the servers 106; a primarypublic network 104 with a private sub-network 104; a primary privatenetwork 104 with a public sub-network 104; or a primary private network104 with a private sub-network 104. According to some embodiments, thenetwork topology of the network 104 differs within differentembodiments, possible network topologies include: a bus networktopology; a star network topology; a ring network topology; arepeater-based network topology; or a tiered-star network topology.Additional embodiments include a network 104 of mobile telephonenetworks that use a protocol to communicate among mobile devices, wherethe protocol is any one of the following: AMPS; TDMA; CDMA; GSM; GPRSUMTS; or any other protocol able to transmit data among mobile devices.

In various embodiments, the computing environment 101 includes more thanone server 106A-106N such that the servers 106A-106N are logicallygrouped together into a server farm 106. In some embodiments, the serverfarm 106 includes servers 106 that are geographically dispersed andlogically grouped together in a server farm 106, servers 106 that arelocated proximate to each other and logically grouped together in aserver farm 106, or several virtual servers executing on physicalservers. Geographically dispersed servers 106A-106N within a server farm106 can, in some embodiments, communicate using a WAN, MAN, or LAN,where different geographic regions can be characterized as: differentcontinents; different regions of a continent; different countries;different states; different cities; different campuses; different rooms;or any combination of the preceding geographical locations. In someembodiments, the server farm 106 is administered as a single entity,while in other embodiments, the server farm 106 includes multiple serverfarms 106.

Referring now to FIG. 2, according to various embodiments of theinvention, information 204 regarding a contextual search is transferredbetween the server(s) 106 and the component(s) 202. According to variousembodiments, the component(s) 202 is computer software. According toother embodiments, the component(s) 202 is computer hardware orcircuitry. According to various embodiments, the component(s) 202 ispart of, or executes on, the client(s) 102. According to someembodiments, the component(s) information 204 is transferred to theserver(s) 106 from the component(s) 202. According to variousembodiments, information 204 is transferred to the component(s) from theserver(s) 106. According to various embodiments, information 204 istransferred back and forth between the server(s) 106 and thecomponent(s) 202.

FIGS. 3A-D illustrate the user experience of the contextual searchservice according to a non-limiting, exemplary embodiment. In FIG. 3A,the user has navigated to a webpage in a web browser using atouch-screen tablet device. In FIG. 3B, contextual mode has beenactivated, and some search entities have been identified on the displayby highlighting certain words on the underlying webpage. In FIG. 3C, theservice has automatically highlighted certain words to notify the userthat more information is available about those terms (although all theitems, words, and objects on the webpage are selectable and searchable).In FIG. 3D, the words “New” and “York” are highlighted, to the right ofthe overlay window. Although those two words are on separate lines, theservice determined that they were related by comparing those words withthe context, and then highlighting them both on the display. A searchwas conducted, which returned the results displayed in the overlaywindow.

In another embodiment, the system does not require activation ofcontextual mode to allow a user to perform a search. In theseembodiment, the user navigates to a web page, as shown in FIG. 3A andany word or series of words displayed by the web page is available forselection by the user. The user indicates a desire to search foradditional information about a term, such as by a activating a specificmouse button or by executing a “long press” on a touch screen, and theterm is highlighted in a manner similar to that shown in FIG. 3D. Insome of these embodiments, the system will highlight entities that arerecognized as potential search terms without input from the user.Selection of terms to automatically highlight can be based on prior useof the system by users (e.g. this term has been searched by multipleusers recently) or can be based on other considerations (e.g.highlighting a currency denomination to educate the user that such aterm can be searched).

FIGS. 4 and 5A-5D show non-limiting processes for a contextual searchservice, according to various embodiments of the invention. Referring toFIG. 4, in step S400, according to some embodiments, the contextualsearching mode is activated by a component(s) 202 before contextualsearches can be performed. According to some embodiments, contextualsearching mode is activated by the contextual search service acceptinginput, using a processor of a computing device, from the user to enableit. Examples include (but are not limited to), user input in the form ofmouse input, touch-screen input, keyboard input, spoken command, or thelike. For instance, in a non-limiting, exemplary embodiment, contextualsearching mode is activated with a touch-screen device by receivinginput corresponding with a sliding movement from the bottom of thescreen. For another example, in a non-limiting, exemplary embodiment,contextual searching mode is activated by receiving input from akeyboard corresponding with a user pressing Alt (or a specific metakey). Once activated, according to some embodiments, the serviceindicates, using a display, audio device, or the like, that thecontextual searching mode has been activated.

In still other embodiments, contextual mode is “always on,” that is; itdoes not need to be activated in order for contextual searches to beperformed. In these embodiments, the user selects a term to be searchedby providing specific, predefined input such as the activation of aspecific mouse button (e.g., right-clicking), the activation of aspecific mouse button in combination with a specific key of a keyboard(e.g., control-click), or by providing a specific touch gesture, such asa double-finger press, triple-finger press or long press. In someembodiments, the user's finger must contact the screen for more than 500milliseconds to register a long press. In other embodiments the lengthof time the user's finger's must contact the screen in order to registera long press is greater than 100 milliseconds, 300 milliseconds, 750milliseconds, or 1000 milliseconds.

According to some embodiments, contextual searching mode is madepersistent (that is, to stay on until it is turned off) when activated,or in some embodiments, is made optionally persistent by accepting inputindicating that a user pressed a hotkey or otherwise indicated that theoption was selected. According to further embodiments, contextualsearching mode can be made non-persistent (when it would otherwise bepersistent) by accepting input indicating that a user pressed a hotkeyor otherwise indicated that that option was selected.

For some embodiments in which contextual mode is activated, enteringcontextual mode involves suspending the otherwise typical user interfacebehavior of the webpage browser, email client, or document viewer. Forembodiments in which contextual mode is “always on,” the activation ofcontextual searching suspends the normal behavior of the page. Accordingto some embodiments, the ability of those software applications toaccept input as normal may be suspended until the service is no longerin contextual mode. In that case, only the service will respond to userinput directed at those applications unless the user input is directedat disabling the service or contextual mode. In these embodiments, userinteraction events with the underlying page are intercepted andredirected or, in some case, discarded. For example, the followingJavaScript event may be used to perform the interception of userinteraction events: event.stopPropagation( ).

In a non-limiting, exemplary embodiment, a computer monitor may displaythe webpage shown in FIG. 7A in a browser window, where the word “JOBS”is hyperlinked. While not in contextual mode, clicking on “JOBS” wouldcause the browser to load the webpage specified by the hyperlink. Whilein contextual mode, clicking on “JOBS” would not cause the browser toload that webpage. Instead, “JOBS” would be highlighted by the service,indicating that “JOBS” is a search entity.

Returning to FIG. 4, in step S410, according to various embodiments, thecontextual search service identifies at least one search entity.According to some embodiments, the search entity (or entities) isidentified by the contextual search service accepting input, using aprocessor of a computing device, from the user to indicate searchterm(s). Examples include (but are not limited to), user input in theform of mouse input, touch-screen input, keyboard input, voice input, orthe like. According to some embodiments, the service accepts input froma mouse corresponding with a user clicking on something. According tosome embodiments, the service accepts input from a touch-screen devicecorresponding with a user touching, pointing, holding, releasing, makinga dragging movement, or the like. In still other embodiments, the usermay fine-tune the selection of a search term by providing additionalinput. For example, the user may drag the leading edge of a userinterface element to indicate a desire to include in their selection aword or words immediately preceding the initially selected search term.

According to some embodiments, the contextual search service determinesthe search entity (or entities) at least in part through adisambiguation process, using a processor of a computing device. Thedisambiguation process itself also involves determining the localcontext in order to automatically select additional search terms or todetermine the boundaries of the search entity (or entities). In someembodiments, the context is used to determine how often variouscombinations of potential terms are each searched. The frequency is thenused, at least in part, to determine which additional search terms, ifany, to select, beyond the one(s) indicated by the user. According tosome embodiments, this disambiguation instead is performed by theserver(s) 106 during step S440. According to some embodiments,disambiguation takes place both during steps S410 and S440.

For example, according to a non-limiting, exemplary embodiment, atouch-screen computer user highlights the word “Depp” while viewing awebpage about the movie, The Pirates of the Caribbean. The contextualsearch service takes as input the terms surrounding Depp, which were,“The actors included Johnny Depp, Keira Knightley, and Orlando Bloom.”The service then determines that, for instance, “Johnny Depp” is a termthat is searched for more often than “Depp, Keira”. The service alsodetermines that “Johnny Depp” is a more desirable search term than“Depp”. Therefore, the service selects “Johnny Depp” as the searchentity.

According to some embodiments, the contextual search service determinesthe local context by finding other term(s), collections of term(s), orthe like, in proximity to those indicated by the user. According to someembodiments, such terms include (but are not limited to): postaladdresses, email addresses, URLs (which may or may not correspond with ahypertext link), currency, phone numbers, geographic locations, or thelike. In these embodiments, regular expressions may be used to determinethe existence of these types of terms. Exemplary regular expressionsinclude: address-[{regex: /[\t\r\n]*(.+[,\u00A0]{1,)[A-Z]{2}[\u00A0]?[\d]{5})($|[\-\r\n]+)/}, {regex: /(\d+\w*[̂\r\n,]+(?:Street|St|Avenue|Ave)(, \d+\w+Floor)?)/}]; currency-{code:‘USD’, symbol: ‘$’, name: ‘dollars?’},{code: ‘EUR’, symbol: ‘

’, name: ‘euros?’},{code: ‘GBP’, symbol: ‘£’, name: ‘pounds?’}; Twitterhandle-/̂@[A-Za-z0-9_]+$/; and phone-[{regex: /([\d]{2} [\d]{2} [\d]{2)[\d]{2) [W]{2})/},{regex: /([\d]{4} [W]{4))/}, regex:/((\+[\d]+)?\(?[\d]{3}\)?[-][\d]{3}[-][\d]{4})/}].

According to some embodiments, search terms may be chosen based onwhether the terms begin with a capital letter, are in all capitalletters, or are in mixed case. According to some embodiments, such termsmay be chosen from the beginning or end of a nearby sentence, where someembodiments take care to ignore periods that do not denote the end of asentence (such as those used for abbreviations such as “U.S.” or“Inc.”). Some embodiments choose such terms based on the existence ornon-existence of a nearby word “bridges” such as (but not limited to)the “of” in “the United States of America”, apostrophes used to indicatepossession (“Gerry's”), ampersands (“His & Hers”), and the like. In manyembodiments this behavior is provided via a rule base containingcommonly-used rules.

Some embodiments ignore terms that are common (for example, ignoring“more”). Some embodiments choose terms based on whether the surroundingHTML tag or other encoding denotes it with a special tag (for example,bold, italic, links, heading tags (<h1>, <h2>, <h3>), or the like). Someembodiments choose terms based on whether the parent node in a HTML orsimilar document is marked as a Microformat or special semantic tag.Some embodiments choose terms based on a Microformat where theMicroformat consists of multiple parts (like an address), scansurrounding areas for the other related tags to highlight them at thesame time. Some embodiments choose terms based on detected sentences,dictionary terms, nouns, or the like. Various embodiments choose termsbased on any combination of the examples described above, or othersimilar examples.

In still further embodiments, every type of analysis described above isperformed in order to identify search terms extant on a displayed page.In still other embodiments, various subsets of the identified techniquesmay be applied. For embodiments in which the client machine is resourceconstrained, such as a tablet or mobile phone, a restricted area isanalyzed to conserve resources. For example, the following JavaScriptmay be used to determine if the device is a mobile device:navigator.userAgent.indexOf(‘mobile’). The rules and techniques may beapplied independently of structural boundaries in the displayed page;for example, processing is not affected by text wrapping from one lineto the next.

In step S420, according to various embodiments, the contextual searchservice notifies, using a processor, the user of the search entity (orentities) identified in step S410. According to some embodiments, thenotification is visual, such as through a display of a computer device.In some embodiments, the search entity (or entities) is highlightedwithin a document on a display. In other embodiments, a copy of theselected area is magnified in order to help the user identify theselected entity. In still further embodiments, the document is overlaidwith a layer to improve contrast of the selected entity, however thatcontrast is achieved. For example, in the embodiment shown in FIGS.3A-3D, a darker layer is used to achieve contrast. In other embodiments,the text style of selected text is analyzed and the replicated with thecolor changed to black for best contrast; in an HTML document thefont-style, font-family, font-weight, and text-transformation styleswill be analyzed and replicated. Alternatively, these styles can bedetermined using the following JavaScript function:node.style.fontFamily.

In some embodiments, a pop-up window, text box, or the like is displayedto the user above the contrast layer (if provided), which may include alist of any search entities. According to some embodiments, the selectedsearch entity (or entities) is the same as what was indicated by theuser, so no additional notification is made. According to someembodiments, the service requests input from the user regarding whichsearch entity to select. In some further embodiments, the serviceaccepts inputs from the user indicating the user's selection. In someembodiments, the input indicates that the user pressed a key, touched atouch screen, pointed with a mouse, made a voice command, or otherwiseindicated the user's selection.

In some other embodiments the selected search entity is an image, video,audio, or other non-textual entity. In these embodiments, a highlight isapplied to the selected entity and it is brought to the foreground, forexample, displayed on top of the contrast layer, if present.

According to some embodiments, the service has already selected thesearch entity, whether or not textual, and requests input from the userto confirm that the user wishes to conduct a search using that searchentity.

In step S430, according to various embodiments, after a search entity isselected, that is, after the touch/click is registered and released, aquery is a string constructed and posted to a server using a processor.According to various embodiments, the rich query string is a URL encodedso that it can be included in the body of a HTTP POST request. The richquery includes both the search entity and context information(hereinafter referred to as a “rich query”). According to someembodiments, the rich query includes context information which increasesthe likelihood of a relevant search. According to some non-limiting,exemplary embodiments, the context information includes any combinationof the title of the page on which the entity is found (accessible usingthe JavaScript command, document.title), the URL of the page on whichthe entity is found (accessible using the JavaScript commanddocument.location or, in some embodiments, read from native code usingprovided APIs), META tags, information regarding word frequencies,and/or the like, of the webpage where the search entity was found. Wordfrequencies may be calculated on the client and included in the HTTPPOST request to the server. According to some embodiments, the contextinformation includes any combination of the text that is located justbefore the search entity, the text that is located just after the searchentity, the corresponding URL if hypertext tags surround the searchentity, information (such as attributes) inside a parent tag surroundingthe search entity, information regarding semantic tags surrounding thesearch entity, information regarding the application currently beingexecuted (such as a web browser), information related to previoussearches, information related to web pages visited, information relatedto hypertext links selected, information related to the user's profile,and/or the like.

Information about the application currently being executed may beexposed by an API provided by the operating system of the client. Insome particular embodiments, the system is able to access the title ofthe window, or window name, in which the selected entity appears as wellas the name of the executable application responsible for displaying theselected entity. Certain applications may allow access to furtherinformation. For example, an email program may also allow access to thetitle of the email, that is, the “subject” line and a word processingprogram may allow access to the title of the displayed document or toother characteristics of the document, such as word count.

Contextual information may be obtained by traversing the DOM tree toascertain the node previous to the node containing the selected entityand extracting the content of the previous node and also to ascertainthe node following the node containing the selected entity andextracting the content of the next node. Thus, in the followingsentence: “Nicolas Sarkozy is the 23^(rd) President of the FrenchRepublic,” in which 23^(rd) President has been selected, the text“Nicolas Sarkozy” will also be extracted and included the HTTP POSTrequest. A similar technique may be used to ascertain text after theselected entity, i.e., in the sentence above the term “French Republic”may also be extracted and included in the HTTP POST request. In someembodiments, this technique is used until 25 characters before theselected entity are obtained, 25 characters after the selected entityare obtained, or both, in other embodiments it is used to obtain 50characters before the selected entity are obtained, 50 characters afterthe selected entity are obtained, or both, in still other embodiments itis used to obtain 100 characters before the selected entity areobtained, 100 characters after the selected entity are obtained, or bothand in still further embodiments it is used to obtain 250 charactersbefore the selected entity are obtained, 250 characters after theselected entity are obtained, or both.

Exemplary code for extracting the content from the previous node is asfollows:

    getPreviousTextNode: function(node) {   var childs =node.parentNode.childNodes,   idx = −1;   for (var i=0; i<childs.length;i++) {     if (childs[i] === node) {       idx = i;       break;     }  }   if (idx > 0) {     var pnode = childs[idx−1];     while(pnode.childNodes.length > 0) {       pnode =pnode.childNodes[pnode.childNodes.length−1];     }     return pnode;   }else {     return this.getPreviousTextNode(node.parentNode);   } }

Similarly, exemplary code for obtaining the content of the next node isas follows:

  getNextTextNode: function(node) { if (!node || !node.parentNode)return null; var childs = node.parentNode.childNodes, idx = −1; for (vari=0; i<childs.length; i++) {   if (childs[i] === node) {     idx = i;    break;   } } if (idx < childs.length−1) {   var pnode =childs[idx+1];   while (pnode.childNodes.length > 0) {     pnode =pnode.childNodes[0];   }   return pnode;   } else {     returnthis.getNextTextNode(node.parentNode);   } }

In some embodiments the selected entity is part of a hyperlink, or link.For example, using the example above, when selecting “23^(rd) President”the user may actually be selecting the following HTML code:

   <div>  Nicolas Sarkozy is the  <a href=’http://www.presidents.fr’title=’Presidents of France’>   23rd President of the French Republic </a> </div>

As described above, the selection of an entity will suspend normalprocessing by the page. Thus, rather than traversing the link identifiedby the text “23^(rd) President,” that selection will be intercepted anddiscarded. The system will then recognize, using the “A” HTML tag thatthe selected entity is a link and will then also extract from the HTMLcode helpful attributes from the link, such as the HREF, ALT and TITLEattributes, in the example above. These extracted attributes areincluded with the selected entity in the HTTP POST request. The systemmay further use the JavaScript command “element.getattribute( )” inorder to acquire the attributes of a tag.

In still other embodiments, the system may acquire text associated withheaders from the page on which the selected entity appears using theJavaScript command“document.querySelectorAll(‘h1,h2,h3,h4,h5,h6,h7.h8’). The acquired textis then included in the HTTP POST request sent to the server. The systemmay also search the DOM of the web page to identify META tags. The textassociated with META tags may also be useful. Each of the META tags in adocument will be returned by the following command:document.getElementsByTagName(‘meta’) and each returned tag may beanalyzed using the following exemplary code:

    for(i = 0; i < metas.length; i++) {   if(metas[i].name ==‘keywords’){     richQuery[‘keywords’] = metas[i].content;   } }

According to a non-limiting, exemplary embodiment, FIG. 7A shows asample webpage and FIG. 7B shows a corresponding rich-query string. Theuser has highlighted “132 Crosby Street, 4^(th) Floor New York, N.Y.10012”, which is used as the search entity. The rich-query stringincludes the name of the server (“query-server-beta”), the data type(“JSON”), a requested layout type (“default”), a return type(“javascript”), a language or culture specifier (“en-US”), and the URLof the search entity's webpage (“http://kikin.com/contact”).Additionally, the rich-query string includes information found on thesearch entity's webpage, including an address (“132 Crosby Street, 4thFloor, New York, N.Y. 10012”), text in close proximity to the address(“kikin Headquarters”), a phone number “646-395-1280”), prominent text(“Contact Us”), and keywords specified in the META tags (“kikin,facebook, twitter, social networking, price comparison, browserplugin”).

In another embodiment, information about the HTML hierarchy of the pageon which the selected entity is found may be used to determine content.In the example above, the HTML code for displaying the address to theuser may be:

   <div id=’address’>  Our address:  <address>   132 crosby street, NewYork, NY 10012  </address> </div>

In these embodiments selecting “New York” will cause the system totraverse the DOM model for the page and detect that the selected entityis part of an ADDRESS tag and a DIV tag. The extracted hierarchalinformation is included in the HTTP POST request sent to, the server.

In still other embodiments, the system may identify entities usingmicroformats, when available. Microformats are attributes of HTMLelements that store extra information, not visible to the user, about atag. For example:

    <p itemprop=“address” itemscope=“”itemtype=“http://data-vocabulary.org/Address”>   <spanitemprop=“stress-address”>2 Lincoln Place</span>,     <spanitemprop=“locality”>       Brooklyn     </span>     <spanitemprop=“region”>       NY     </span>   </span> </p>

In the example above if the user selects “Brooklyn,” the system willdetect the presence of the property “itemprop” and parse the DOM tree totag the selection as an address. If a microformat is identified, it issaved and included in the HTTP POST request to the server.

According to various embodiments, information related the user's profilemay also be used when formulating a rich query for transmission to theserver. Such information includes, but is not limited to, anycombination of the user's history of documents visited, favoritewebsites or bookmarks, categories of favorite websites or bookmarks,shares, prior searches, recent prior searches, the search that referredthe user to the current webpage, saves, interests, social networks,frequently accessed applications/documents/websites, a Bloom filter datastructure that represents all domains that the user visited during aspecific period of time, a Bloom filter data structure that representsall domains from which the user saved a webpage, a Bloom filter datastructure that represents all domains on which the user shared awebpage, or the like. According to a non-limiting, exemplary embodiment,FIG. 6 shows a textual representation of a data structure containinginformation regarding a sample user profile.

Prior searches performed by the user are stored and may be used as partof the user profile. Additionally, as the user visits web pages, theURLs of those web pages are recorded and sent to the server. Forexample, is a user searches www.google.com for “new camera” and thenvisits multiple web pages about cameras, that search and the pagesvisited by the user will be sent to the server and used to refine acontextual query to direct it more towards camera-related results.

Similarly, the bookmarks saved by a user during web browsing areaccessed and sent to the server to be used to refine a contextualsearch. For example, if a user has bookmarked google.com, bbc.com/news,and cnn.com, the server may use this fact to provide more news-relatedsites in response to a request.

According to various embodiments, after the rich query is constructed,this rich query is sent, using a processor, from the component(s) 202 tothe server(s) 106. According to some embodiments, more than one richquery may be constructed and sent. According to further embodiments, therich queries may be identical, and in other embodiments, at least someof the rich queries may be at least partially different from anotherrich query. According to various embodiments, the processor sends therich query (or queries) to the server(s) 106 using a networking device.According to some embodiments, the rich query is not sent to a serverand is instead processed on the client 102 by the component(s) 202Alternatively, the rich query may be both sent to the server(s) 106 bythe client and processed locally by component(s) 202. According tofurther embodiments, no rich query is constructed, and instead thecontextual information that would have otherwise been used for a richquery is sent to the server(s) 106, processed directly by thecomponent(s) 202, or both.

In step S440, according to various embodiments, the contextual searchservice, using a processor, obtains the context from the rich query (orqueries) it receives from the component(s) 202. According to someembodiments, the service preprocesses the attributes stored in the richquery (or queries) to identify context that is potentially useful toimprove search results. According to some embodiments, the servicecategorizes the search entity (or entities) based on the contextinformation stored in the search query (or queries). According to someembodiments, the service disambiguates the meaning of any ambiguousterms with the help of the context information. The service may alsoleverage the results obtained by other users of the service to furtherrefine the results given to any particular user.

For embodiments in which the selected entity is an image, the image issubmitted to a facial recognition service to identify the picturedperson or product. The result from that conversion, which is text, isthen used by the service as the selected entity to search. Forembodiments in which the selected entity is audio, a digital fingerprintof the audio is submitted to an audio recognition service, such asservices offered by Rovi Shazam and IntoNow. In some embodiments, theaudio fingerprint that is submitted to the audio recognition service isgenerated by executing a hash on the PCM data comprising the file. Theresult returned from the audio recognition service, which is text, isthen used as the selected entity to search.

For embodiments in which the selected entity is textual, the serveridentifies a most probable sequence of words based on the selected textand the surrounding text. In one embodiment, the server accomplishesthis by generating a list of n-word-grams. In some embodiments, thenumber words used to create the word grams are 10 words, in otherembodiments 8 words are used, in still other embodiments 6 words areused, in still further embodiments 4 words are used and in still otherembodiments 2 words are used.

As an example of this, for the sentence:

Tunisia, the country where the Arab Spring uprisings began this year,has joined the International Criminal court, becoming the first NorthAfrican country to do so

in which “International Criminal” was the selected text would result, inan embodiment in which 6 words are used, in the following set ofword-grams:

-   this year has joined the international-   year has joined the international criminal-   has joined the international criminal court-   joined the international criminal court becoming-   the international criminal court becoming the-   international criminal court becoming the first-   criminal court becoming the first north-   year has joined the international-   has joined the international criminal-   joined the international criminal court-   the international criminal court becoming-   international criminal court becoming the-   criminal court becoming the first-   has joined the international-   joined the international criminal-   the international criminal court-   international criminal court becoming-   criminal court becoming the-   joined the international-   the international criminal-   international criminal court-   criminal court becoming-   the international-   international criminal-   criminal court-   international-   criminal

In some embodiments, each of the generated word-grams may be used asinput to a search engine, such as google.com, bing.com, yahoo.com andthe like. The results returned from these searches can be reviewed todetermine if any of the results contain links to a knowledge managementsite, such as wikipedia.com. Word-grams that returned search resultshaving links to a knowledge management site are then ordered from mostpopular to least popular. The longest word-gram returning the mostpopular search is selected as the intended search term.

The selected search term may then be reviewed to determine if the wordor word sequence has multiple meanings. In one embodiment, this isaccomplished by consulting a directory of ambiguous phrases, such asthat maintained by wikipedia.com. In this embodiment, if the phrase isambiguous, compare the text of the page where the user originallyselected the text, with the text on each page that is linked from thedisambiguation page. Select the page with the most significant overlapand adjust the highlight text based on this. In a specific example, forthe following text: https://www.applebank.com/about.aspx when the userhighlights “Apple,” a request is made to a disambiguation page, e.g.,http://en.wikipedia.org/wiki/Apple_(disambiguation). The text on theabout page is compared with each page linked from above disambiguationpage. Because words used on http://en.wikipedia.org/wiki/Apple_Bankoverlap more than with the other pages, e.g., 1863 (the year the bankwas incorporated), 125^(th) (the street the first branch was locatedon), mortgage, financial, or New York, Apple Bank is selected as thedisambiguated search term to use going forward.

In step S450, according to various embodiments, the contextual searchservice, using a processor, identifies relevant content source(s) forquery submission. Content sources include, but are not limited to, theInternet, specific commerce, travel, or other service-related sources,knowledge sources such as Wikipedia and Wolfram Alpha, proprietarysources such as corporate databases, and the like. According to variousembodiments, the contextual information determined in step S440 is usedto determine which content source(s) is (or are) relevant. For example,according to some non-limiting exemplary embodiments, rich queries whichappear to relate to travel destinations (according to the search entityand its context) will be matched by the service to the Internet andtravel-related sources. According to some embodiments, the contentsource(s) is (or are) determined based at least in part on the type ofmedia that the rich query seems to be related to, for example, videos,photos, etc. According to some embodiments, the content source(s) is (orare) determined based at least in part on the ontological category thatthe rich query seems to be related to, for example, commerce, travel,etc. According to some embodiments, preferences indicated by the userand received by the service determine, in part, the type of contentsource(s).

In more detail, step S450 may be performed by searching groupings ofcontent sources, referred to herein as “vertical categories.” Ingeneral, the disambiguated word or word sequence is used to returngeneral search results which are then, in turn, compared to a verticalcategory mapping directory to identify potentially relevant contentsources. In some embodiments, the vertical categories may be selected orprioritized based on, for example, context-specific skew or userpreferences.

A specific example of this process is how content sources are selectedfor a general search term, such as “Hawaii.” A general search for“Hawaii” may return, for example, the following results:

-   1. www.gohawaii.com-   2. www.gohawaii.com/big-island-   3. Map of Hawaii-   4. hawaii.gov/5.-   www.capitol.hawaii.gov/session2011/-   6. Images for Hawaii-   7. en.wikipedia.org/wiki/Hawaii-   8. www.co.hawaii.hi.us/-   9. www.lonelyplanet.com/usa/hawaii

A URL prefix to category index contains the categorization for“Map->Map”, “Images->Images”, “Wikipedia->References”, and “LonelyPlanet->Travel”. Based on these results, the serve selects the followingvertical categorization: Map, Images, References, and Travel. In oneembodiment, for each domain, the verticals most frequently accessed bythe user are tracked. That is, for example, if people onlonelyplanet.com happen to view more Images then Maps, then images willbe prioritized over maps for all users in future searches associatedwith lonelyplanet.com. However if a specific user watches videos withgreater frequency than the general population, the video category willbe introduced (or boosted) only for that user.

In step S460, according to various embodiments, the contextual searchservice, using a processor, converts the rich query (or queries) to astandard query (or queries). Standard queries are queries that areacceptable as input to the content source(s) identified in step S450.According to various embodiments, a standard query includes one or moresearch entities. According to various embodiments, a standard queryadditionally includes context information related to the search entity.According to some embodiments, if during step S440, the servicedisambiguated the meaning of any ambiguous terms with the help of thecontextual information, additional terms may be included in the standardquery, as determined by the disambiguation process. According to someembodiments, Boolean logic is applied to standard query construction. Ina non-limiting embodiment, for example, Boolean logic is used inconstructing a standard query for an Internet Search source whichaccepts Boolean search queries.

According to some embodiments, unnecessary words in the search entityare not included in the standard query. For example, if the searchentity is long, the search entity may be shortened for the standardquery, to increase the diversity of results. This is because searchstrings that are too specific may return results from too narrow a listof sources.

In greater detail, for each vertical source identified, a query stringis created that can be understood by the context source. In someembodiments, each vertical source has an associated input and outputmapping, which translates a rich query received from a client intospecific instructions for the content source. For example, to searchYouTube the following query instruction may be generated:

http://gdata.youtube.com/feeds/api/videos?q={QUERY}&orderby=relevance&safeSearch=moderate&max-results=20&format=5&v=2&client=ytapi-kikin-Youtubevideowidgs7nshfa6-0&lr={LANGUAGE}The placeholders {QUERY} and {LANGUAGE} would be populated based on therich query. YouTube results may be mapped in the following manner:

-   feed/entries/title->title-   feed/entries/links/href->link-   feed/entries/mediaGroup/thumbnails->thumbnail    in order to return result that can be used by the service. In some    embodiments, all content sources are accessed in parallel and the    results of each source are post-filtered and re-ordered.

In step S470, according to various embodiments, the contextual searchservice, using a processor, obtains results from the content source(s).According to various embodiments, the results are obtained by submittingthe standard query (or queries) that were constructed in step S460 tothe content source(s) identified in step S450, and the service thenaccepts results from one or more of the content source(s), According tovarious embodiments, the results are processed. According to someembodiments, the results are sorted to remove duplicates (for example,sending results to multiple content sources may return results from onecontent source that duplicate one or more of the results from anothercontent source). According to some embodiments, the results are sortedin order of contextual relevance. According to various embodiments, theresults or processed results are sent to the component(s) 202 by aprocessor, using a networking device.

According to a non-limiting, exemplary embodiment, FIG. 7C shows atextual representation of a data structure containing sample resultsreturned by a server that had been provided the rich query string ofFIG. 7B. The search entity was “132 Crosby Street, 4th Floor New York,N.Y. 10012”. Information regarding the user's profile included a countrycode, IP address, country, region, city, DMA code, region, area code,postal code, continent code, latitude and longitude. The resultsincluded map information regarding address provided in the search entity(latitude, longitude, and postal address). The results further includeda website related to the location of the address provided in the searchentity (the local government website).

Returning to FIG. 4, in step S480, according to various embodiments, thecontextual search service, using a processor, filters the results.According to some embodiments, the results are filtered according tosettings received from the user or stored in a user profile. Accordingto some embodiments, the results are filtered according to the physicallimitations of the device of the components (202), for example, thedevice screen size, latency, and/or caching.

In greater detail, the results of all content source searched may bemerged based on the order of the vertical categories and the mergedresults may be sent to the client. In some embodiments, the results aresent to client together with client-side query descriptors. In somespecific embodiments, the search results are sent to the client in aJSON-encoded structure.

In step S490, according to various embodiments, the contextual searchservice, using a processor, presents the results. According to variousembodiments, presenting the results involves rendering the results on adisplay. According to some embodiments, the service accepts input fromthe user to update the search entity so that a new search may beconducted. According to some embodiments, the results are displayed in apop-up window, dialogue box, unopened or opened tab, overlay, or thelike. According to some embodiments, the results are displayed in textform and/or rich media form, including images, videos, maps, and/or thelike. In still other embodiments, the server provides the querytranslation to the client and the client itself executes the query.

In some embodiments, the client may itself perform the same series ofsteps described above, either independently or in parallel. For example,the client may execute parallel queries of content sources and mergethose results into the results received from the server. For theseembodiments, the client follows the same query logic as that describedabove in connection with the server. In certain embodiments, the resultsfrom the server may be used to update or adjust the selected entity. Inthese embodiments, the updated or adjusted selected entity may besearched, as described above.

Having described certain embodiments of an apparatus for interactingwith software, it will now become apparent to one of skill in the artthat other embodiments incorporating the concepts of the disclosure maybe used.

1.-20. (canceled)
 21. A method for constructing a rich query from aselected semantic entity included in a page, the method comprising:receiving user input identifying a semantic entity, from a plurality ofsemantic entities, for which contextual information is desired;identifying context information associated with the selected semanticentity from the plurality of semantic entities; encoding a UniformResource Locator (URL) using the semantic entity and the identifiedcontext information; and transmitting the encoded URL to a searchserver.
 22. The method of claim 21, wherein user input is received via atouch screen interface.
 23. The method of claim 21, wherein contextinformation is identified using characteristics of the page in which thesemantic entity was included.
 24. The method of claim 23, whereincontext information is identified using one of a title of a web page, aURL identifying a web page, a META tag included in a web page, andinformation identifying word frequencies of the web page including thesemantic entity.
 25. The method of claim 21, wherein context informationis identified using information regarding the user's browsing history.26. The method of claim 25, wherein the information regarding the user'sbrowsing history includes one of the user's browsing history, favoriteweb sites of the user, bookmarks saved by the user, prior searchesrequested by the user, and frequently accessed web sites.
 27. The methodof claim 21, wherein context information is identified using a profileestablished by the user.
 28. The method of claim 21, wherein contextinformation is identified using a Bloom filter.
 29. The method of claim21, further comprising encoding a first URL using the semantic entityand first context information and a second URL using the semantic entityand second context information.
 30. The method of claim 21, wherein theencoded URL is transmitted using an HTTP POST command.
 31. A system forconstructing a rich query from a selected semantic entity included in apage, the system comprising: a processor coupled to a computer readablemedium storing processor executable instructions that when executed bythe processor causes the processor to: receive a user input identifyinga semantic entity, from a plurality of semantic entities, for whichcontextual information is desired; identify context informationassociated with the selected semantic entity from the plurality ofsemantic entities; encode a Uniform Resource Locator (URL) using thesemantic entity an the identified context information; and transmit theencoded URL to a search server.
 32. The system of claim 31, whereinexecution of the processor executable instructions further causes theprocessor to receive user input via a touch screen interface.
 33. Thesystem of claim 31, wherein execution of the processor executableinstructions further causes the processor to identify contextualinformation using characteristics of the page in which the semanticentity was included.
 34. The system of claim 31, wherein execution ofthe processor executable instructions further causes the processor toidentify contextual information using one of a title of a web page, aURL identifying a web page, a META tag included in a web page, andinformation identifying word frequencies of the web page including thesemantic entity.
 35. The system of claim 31, wherein execution of theprocessor executable instructions further causes the processor toidentify contextual information using information regarding the user'sbrowsing history.
 36. The system of claim 35, wherein the informationregarding the user's browsing history includes one of the user'sbrowsing history, favorite web sites of the user, bookmarks saved by theuser, prior searches requested by the user, and frequently accessed websites.
 37. The system of claim 31, wherein execution of the processorexecutable instructions further causes the processor to identifycontextual information using a profile established by the user.
 38. Thesystem of claim 31, wherein execution of the processor executableinstructions further causes the processor to identify contextualinformation using a Bloom filter.
 39. The system of claim 31, whereinexecution of the processor executable instructions further causes theprocessor to encode a first URL using the semantic entity and firstcontext information and a second URL using the semantic entity andsecond context information.
 40. The system of claim 31, whereinexecution of the processor executable instructions further causes theprocessor to transmit the encoded URL using an HTTP POST command.